<!DOCTYPE html>
<html lang="zh-CN">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>百万级数据导入最佳实践</title>
    <link href="https://cdn.staticfile.org/font-awesome/6.4.0/css/all.min.css" rel="stylesheet">
    <link href="https://cdn.staticfile.org/tailwindcss/2.2.19/tailwind.min.css" rel="stylesheet">
    <link href="https://fonts.googleapis.com/css2?family=Noto+Serif+SC:wght@400;500;600;700&family=Noto+Sans+SC:wght@300;400;500;700&display=swap" rel="stylesheet">
    <script src="https://cdn.jsdelivr.net/npm/mermaid@latest/dist/mermaid.min.js"></script>
    <style>
        body {
            font-family: 'Noto Sans SC', Tahoma, Arial, Roboto, "Droid Sans", "Helvetica Neue", "Droid Sans Fallback", "Heiti SC", "Hiragino Sans GB", Simsun, sans-serif;
        }
        .hero-gradient {
            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
        }
        .card-hover {
            transition: all 0.3s ease;
        }
        .card-hover:hover {
            transform: translateY(-5px);
            box-shadow: 0 20px 40px rgba(0,0,0,0.1);
        }
        .section-title {
            position: relative;
            padding-left: 20px;
        }
        .section-title::before {
            content: '';
            position: absolute;
            left: 0;
            top: 50%;
            transform: translateY(-50%);
            width: 4px;
            height: 30px;
            background: linear-gradient(to bottom, #667eea, #764ba2);
        }
        .drop-cap {
            float: left;
            font-size: 4rem;
            line-height: 1;
            font-weight: 700;
            margin-right: 8px;
            margin-top: -8px;
            color: #667eea;
        }
        .mermaid {
            display: flex;
            justify-content: center;
            margin: 2rem 0;
        }
        .icon-box {
            width: 60px;
            height: 60px;
            display: flex;
            align-items: center;
            justify-content: center;
            border-radius: 12px;
            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
            color: white;
            font-size: 24px;
            margin-bottom: 1rem;
        }
        .highlight-text {
            background: linear-gradient(to right, #667eea, #764ba2);
            -webkit-background-clip: text;
            -webkit-text-fill-color: transparent;
            font-weight: 600;
        }
    </style>
</head>
<body class="bg-gray-50">
    <!-- Hero Section -->
    <section class="hero-gradient text-white py-20 px-6">
        <div class="max-w-6xl mx-auto">
            <div class="text-center">
                <h1 class="text-5xl md:text-6xl font-bold mb-6 leading-tight">
                    百万级数据导入
                    <span class="block text-3xl md:text-4xl mt-2 font-light opacity-90">最佳实践指南</span>
                </h1>
                <p class="text-xl md:text-2xl font-light max-w-3xl mx-auto opacity-90 leading-relaxed">
                    掌握大规模数据处理的艺术，确保系统稳定性与数据完整性
                </p>
            </div>
        </div>
    </section>

    <!-- Introduction -->
    <section class="py-16 px-6">
        <div class="max-w-4xl mx-auto">
            <div class="bg-white rounded-2xl shadow-xl p-8 md:p-12">
                <p class="text-lg leading-relaxed text-gray-700">
                    <span class="drop-cap">在</span>处理百万级数据导入到数据库的过程中，涉及到的数据量巨大，操作的复杂性和对系统资源的需求也随之增加。这类任务不仅对数据库的性能提出了挑战，还可能影响整个系统的稳定性。正确的处理方式对于确保数据完整性、提高导入效率以及减少对现有系统的影响至关重要。
                </p>
            </div>
        </div>
    </section>

    <!-- Data Flow Visualization -->
    <section class="py-12 px-6">
        <div class="max-w-6xl mx-auto">
            <h2 class="text-3xl font-bold text-center mb-12 text-gray-800">数据导入流程架构</h2>
            <div class="bg-white rounded-2xl shadow-lg p-8">
                <div class="mermaid">
                    graph LR
                        A[Excel数据源] --> B[数据预处理]
                        B --> C[批量处理]
                        C --> D[性能优化]
                        D --> E[资源管理]
                        E --> F[错误处理]
                        F --> G[并发处理]
                        G --> H[数据库]
                        
                        style A fill:#f9f,stroke:#333,stroke-width:2px
                        style H fill:#9f9,stroke:#333,stroke-width:2px
                        style B fill:#ffd,stroke:#333,stroke-width:2px
                        style C fill:#ffd,stroke:#333,stroke-width:2px
                        style D fill:#ffd,stroke:#333,stroke-width:2px
                        style E fill:#ffd,stroke:#333,stroke-width:2px
                        style F fill:#ffd,stroke:#333,stroke-width:2px
                        style G fill:#ffd,stroke:#333,stroke-width:2px
                </div>
            </div>
        </div>
    </section>

    <!-- Main Content -->
    <section class="py-16 px-6">
        <div class="max-w-6xl mx-auto">
            <div class="grid grid-cols-1 md:grid-cols-2 gap-8">
                
                <!-- 数据预处理 -->
                <div class="bg-white rounded-2xl shadow-lg p-8 card-hover">
                    <div class="icon-box">
                        <i class="fas fa-filter"></i>
                    </div>
                    <h3 class="section-title text-2xl font-bold mb-6 text-gray-800">数据预处理</h3>
                    <div class="space-y-4">
                        <div>
                            <h4 class="font-semibold text-lg mb-2 text-gray-700">
                                <i class="fas fa-broom text-purple-600 mr-2"></i>数据清洗
                            </h4>
                            <p class="text-gray-600 leading-relaxed">确保导入的数据没有错误或重复记录，这可以减少数据处理中的异常。</p>
                        </div>
                        <div>
                            <h4 class="font-semibold text-lg mb-2 text-gray-700">
                                <i class="fas fa-exchange-alt text-purple-600 mr-2"></i>数据格式
                            </h4>
                            <p class="text-gray-600 leading-relaxed">将 Excel 文件的数据格式转化为数据库能够接受的格式，比如将日期格式转换为标准日期格式。</p>
                        </div>
                    </div>
                </div>

                <!-- 批量处理 -->
                <div class="bg-white rounded-2xl shadow-lg p-8 card-hover">
                    <div class="icon-box">
                        <i class="fas fa-layer-group"></i>
                    </div>
                    <h3 class="section-title text-2xl font-bold mb-6 text-gray-800">批量处理</h3>
                    <div class="space-y-4">
                        <div>
                            <h4 class="font-semibold text-lg mb-2 text-gray-700">
                                <i class="fas fa-database text-purple-600 mr-2"></i>批量插入
                            </h4>
                            <p class="text-gray-600 leading-relaxed">使用批量插入的方式而不是逐条插入，这可以显著提高导入速度。例如，JDBC 提供的 <code class="bg-purple-100 px-2 py-1 rounded text-purple-700">addBatch</code> 和 <code class="bg-purple-100 px-2 py-1 rounded text-purple-700">executeBatch</code> 方法。</p>
                        </div>
                        <div>
                            <h4 class="font-semibold text-lg mb-2 text-gray-700">
                                <i class="fas fa-cubes text-purple-600 mr-2"></i>分批导入
                            </h4>
                            <p class="text-gray-600 leading-relaxed">将数据分成多个小批次进行导入，每个批次处理的记录数应根据系统负载和数据库能力进行调整。</p>
                        </div>
                    </div>
                </div>

                <!-- 性能优化 -->
                <div class="bg-white rounded-2xl shadow-lg p-8 card-hover">
                    <div class="icon-box">
                        <i class="fas fa-tachometer-alt"></i>
                    </div>
                    <h3 class="section-title text-2xl font-bold mb-6 text-gray-800">性能优化</h3>
                    <div class="space-y-4">
                        <div>
                            <h4 class="font-semibold text-lg mb-2 text-gray-700">
                                <i class="fas fa-lock text-purple-600 mr-2"></i>事务管理
                            </h4>
                            <p class="text-gray-600 leading-relaxed">将大规模的数据导入操作放在一个事务中，可以确保数据的一致性。需注意较大事务可能导致长时间锁定。</p>
                        </div>
                        <div>
                            <h4 class="font-semibold text-lg mb-2 text-gray-700">
                                <i class="fas fa-list-ol text-purple-600 mr-2"></i>索引管理
                            </h4>
                            <p class="text-gray-600 leading-relaxed">在数据导入前暂时禁用数据库表的索引，导入完成后再重新启用，这可以提高导入效率。</p>
                        </div>
                    </div>
                </div>

                <!-- 资源管理 -->
                <div class="bg-white rounded-2xl shadow-lg p-8 card-hover">
                    <div class="icon-box">
                        <i class="fas fa-server"></i>
                    </div>
                    <h3 class="section-title text-2xl font-bold mb-6 text-gray-800">资源管理</h3>
                    <div class="space-y-4">
                        <div>
                            <h4 class="font-semibold text-lg mb-2 text-gray-700">
                                <i class="fas fa-memory text-purple-600 mr-2"></i>内存管理
                            </h4>
                            <p class="text-gray-600 leading-relaxed">确保 JVM 的内存设置足够以避免内存溢出。通过设置合理的堆内存大小和垃圾回收策略来优化。</p>
                        </div>
                        <div>
                            <h4 class="font-semibold text-lg mb-2 text-gray-700">
                                <i class="fas fa-network-wired text-purple-600 mr-2"></i>数据库连接池
                            </h4>
                            <p class="text-gray-600 leading-relaxed">配置适当的数据库连接池参数，以处理高并发的数据库连接请求，避免连接池瓶颈。</p>
                        </div>
                    </div>
                </div>

                <!-- 错误处理和日志记录 -->
                <div class="bg-white rounded-2xl shadow-lg p-8 card-hover">
                    <div class="icon-box">
                        <i class="fas fa-shield-alt"></i>
                    </div>
                    <h3 class="section-title text-2xl font-bold mb-6 text-gray-800">错误处理和日志记录</h3>
                    <div class="space-y-4">
                        <div>
                            <h4 class="font-semibold text-lg mb-2 text-gray-700">
                                <i class="fas fa-undo text-purple-600 mr-2"></i>错误回滚
                            </h4>
                            <p class="text-gray-600 leading-relaxed">确保系统能正确处理导入过程中发生的错误，避免部分数据成功导入而部分失败。</p>
                        </div>
                        <div>
                            <h4 class="font-semibold text-lg mb-2 text-gray-700">
                                <i class="fas fa-file-alt text-purple-600 mr-2"></i>日志记录
                            </h4>
                            <p class="text-gray-600 leading-relaxed">记录详细的日志信息，包括导入的进度、成功和失败的记录等，便于问题追踪。</p>
                        </div>
                    </div>
                </div>

                <!-- 并发处理 -->
                <div class="bg-white rounded-2xl shadow-lg p-8 card-hover">
                    <div class="icon-box">
                        <i class="fas fa-sync-alt"></i>
                    </div>
                    <h3 class="section-title text-2xl font-bold mb-6 text-gray-800">并发处理</h3>
                    <div class="space-y-4">
                        <div>
                            <h4 class="font-semibold text-lg mb-2 text-gray-700">
                                <i class="fas fa-tasks text-purple-600 mr-2"></i>并发导入
                            </h4>
                            <p class="text-gray-600 leading-relaxed">如果系统资源允许，可以考虑并发导入多个数据文件或数据块。需确保并发操作不会对数据库造成过大压力。